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ABSTRACT 

The  current  state  and  trajectory  of  development  for  display  technologies  supporting  information  acquisition,  analysis  and 
dissemination  lends  a  broad  informational  infrastructure  to  operators  of  complex  systems.  The  amount  of  information 
available  threatens  to  outstrip  the  perceptual-cognitive  capacities  of  intended  operators,  thus  limiting  their  ability  to 
effectively  interact  with  targeted  technologies.  Therefore,  a  critical  step  in  designing  complex  display  systems  is  to  find 
an  appropriate  match  between  capabilities,  operational  needs,  and  human  ability  to  utilize  complex  information.  The 
present  work  examines  a  set  of  evaluation  parameters  that  were  developed  to  facilitate  the  design  of  systems  to  support  a 
specific  military  need;  that  is,  the  capacity  to  support  the  achievement  and  maintenance  of  real-time  360°  situational 
awareness  (SA)  across  a  range  of  complex  military  environments.  The  focal  point  of  this  evaluation  is  on  the  reciprocity 
native  to  advanced  engineering  and  human  factors  practices,  with  a  specific  emphasis  on  aligning  the  operator-system- 
environment  fit.  That  is,  the  objective  is  to  assess  parameters  for  evaluation  of  360°  SA  display  systems  that  are  suitable 
for  military  operations  in  tactical  platforms  across  a  broad  range  of  current  and  potential  operational  environments.  The 
approach  is  centered  on  five  "families"  of  parameters,  including  vehicle  sensors,  data  transmission,  in-vehicle  displays, 
intelligent  automation,  and  neuroergonomic  considerations.  Parameters  are  examined  under  the  assumption  that  displays 
designed  to  conform  to  natural  neurocognitive  processing  will  enhance  and  stabilize  Soldier-system  performance  and, 
ultimately,  unleash  the  human's  potential  to  actively  achieve  and  maintain  the  awareness  necessary  to  enhance  lethality 
and  survivability  within  modern  and  future  operational  contexts. 

Keywords:  Situational  Awareness,  Indirect-vision  displays.  Systems  engineering,  Neuroergonomics,  Military 
technology 


1.  INTRODUCTION 

Modern  advances  in  the  development  of  display  technologies  supporting  information  acquisition,  analysis  and 
dissemination  lend  a  broad  informational  infrastructure  to  operators  of  complex  systems.  Indeed,  access  to  information, 
especially  the  ability  to  access  the  precise  information  that  is  needed  at  the  moment  that  it  is  needed,  can  be  considered 
one  of  the  critical  factors  underlying  the  performance  of  operations  involving  complex  human-system  interactions.  The 
present  paper  provides  a  conceptual  discussion  of  system  design  for  supporting  complex  human  cognitive  performance 
in  challenging  and  dynamic  environments.  In  specific,  we  examine  a  set  of  evaluation  parameters  that  were  previously 
outlined  as  a  framework  for  assessing  the  utility  of  advanced  systems  supporting  full-spectrum  situational  awareness 
(SA)  in  military  environments1.  Our  aim  is  to  incorporate  greater  consideration  of  human  factors  issues  that,  if  left 
unconsidered,  may  undermine,  or  at  minimum  restrict,  system  utility. 

Without  question,  the  technological  infrastructure  that  supports  operations  that  involve  high  information  density  is 
advancing  at  an  incredible  pace.  Such  advances  are  rapidly  forthcoming  on  both  ends  of  the  informational  continuum. 
That  is,  as  new  developments  in  both  sensor  and  display  systems  become  available  with  regular  frequency.  Examples  of 
modern  display  technologies  that  have  made  their  way  into  the  consumer  markets  include  touch  (and  multi-touch) 
enabled  screens,  ultra-thin  LED  or  plasma-based  monitors,  and  graphics  cards  of  increasing  capability  enabling 
simultaneous  control  over  multiple  graphics-intensive  displays.  Likewise,  sensor  technologies,  particularly  those 
designed  to  facilitate  human-system  interaction  (for  example,  eye  and  body-motion  sensing  technologies  for  control  of 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1 .  REPORT  DATE  2.  REPORT  TYPE 

28  APR  2011  N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Accounting  for  human  neurocognitive  function  in  the  design  and 

5b.  GRANT  NUMBER 

evaluation  oi  jou  degree  situational  awareness  display  system 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

Jason  S.  Medcalfe;  Thomas  Mikulski;  Scott  Dittman 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

US  Army  RDECOM-TARDEC  6501  Ell  Mile  Rd  Warren,  MI 
48397-5000,  USA  DCS  Corporation,  6909  Metro  Park  Dr,  Suite 
500,  Alexandria,  VA  22310  Vorteil  Technologies,  Inc.  116 

Dittman  Ln.,  Karns  City,  PA  16041 

8.  PERFORMING  ORGANIZATION  REPORT  NUMBER 

21665 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

US  Army  RDECOM-TARDEC  6501  Ell  Mile  Rd  Warren,  MI 

10.  SPONSOR/MONITOR’S  ACRONYM! S) 

TACOM/TARDEC/RDECOM 

48397-5000,  USA 

11.  SPONSOR/MONITOR’S  REPORT  NUMBER(S) 

21665 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

Presented  at  SPIE  25-29  April  2011  Orlando,  Florida,  USA,  The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION 

18.  19a.  NAME  OF  RESPONSIBLE  PERSON 

vjr  i  vjdo  i  tvi  vu  i 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  S  AR 

unclassified  unclassified  unclassified 

OF  PAGES 

14 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


UNCLASSIFIED 

gaming  and  other  computing  applications2'3),  are  becoming  increasingly  advanced  and  integral  to  ongoing  technology 
development  efforts. 

Such  technical  achievements  in  sensor  and  display  technologies  are  currently  being  leveraged  to  increase  operator 
productivity  and  safety  as  they  work  on  complex  tasks  within  dynamic  and  often  risky  operational  environments.  One 
such  environment,  the  military  battlefield,  is  especially  in  need  of  these  advanced  technologies.  In  particular,  the  ability 
to  maintain  local  area  security  is  considered  critical  for  the  modern  Warfighter,  who  must  conduct  complex  mobile 
operations  across  a  wide  variety  of  operational  environments.  Of  particular  concern  here  is  the  concept  of  secure 
mobility 4;  that  is,  unlike  civilian  mobility.  Warfighters  must  maintain  local  area  security  through  scanning  of  the 
environment  and  monitoring  various  communications  and  sensor  feeds  while  simultaneously  carrying  out  other  mission- 
critical  tasks.  Moreover,  because  of  the  nature  of  the  threats  imposed  by  adaptive  enemies  that  employ  unconventional 
weaponry  (such  as  improvised  explosive  devices,  or  IEDs),  many  current  and  future-oriented  U.S.  military  development 
efforts  are  being  designed  for  Warfighters  to  conduct  operations  from  within  heavily  armored  vehicles,  including  some 
that  are  completely  enclosed.  As  such,  the  task  of  maintaining  local  area  security  is  made  more  challenging  by  the  fact 
that  Warfighters  often  must  achieve  and  maintain  their  situational  awareness  (SA)  by  indirect  means  (i.e.  cameras)  as 
opposed  to  direct  immersion  in  the  operational  environment  (i.e.  with  their  "head  out  of  the  hatch”). 

2.  U.S.  ARMY  TECHNOLOGY  DEVELOPMENT 

In  response  to  the  challenges  described  above,  the  U.S.  Army  has  been  modernizing  systems  aimed  at  supporting 
enhanced  SA.  It  is  thought  that  such  enhanced  Intelligence,  Surveillance,  and  Reconnaissance  capabilities  with  advanced 
sensors  and  systems  will  serve  a  fundamental  role  toward  the  facilitation  of  achieving  and  maintaining  SA  during  the 
execution  of  future  operations,  particularly  in  complex  urban  settings5.  Specifically,  the  ability  to  sustain  real-time,  full- 
spectrum  SA  is  thought  to  be  facilitated  by  technologies  that  provide  for  hemispherical  (360°/90°;  horizontal/vertical) 
visualization  of  the  surrounding  battlespace  that  is  complimented  by  flexible  high-resolution,  narrow  field  of  view 
sensors  that  can  be  slewed  to  any  position  in  space  near-instantaneously6'7. 

To  date,  the  U.S.  military  has  designed  a  variety  of  360°  SA  systems  for  a  number  of  both  prototype  and  fielded  ground 
vehicles.  For  example,  in  2008  the  U.S.  Army  Communications -Electronics  Research,  Development  and  Engineering 
Center  -  Night  Vision  and  Electronic  Sensors  Directorate  (CERDEC  -  NVESD)  worked  with  industry  to  develop  a 
360°/90°  hemispherical  vision  system  for  the  M2  Bradley.  This  Distributed  Aperture  System,  as  it  was  named,  included 
thirty-three  color  day,  image  intensified,  un-cooled  infrared  sensors  whose  images  were  de-warped,  stitched,  and  fused 
in  real-time  and  then  sent  to  three  independent  in-vehicle  displays.  Warfighters  indicated  that  the  system  increased  their 
SA  as  compared  to  the  baseline  Bradley;  but  ultimately,  large-scale  production  was  not  pursued  because  the  number  and 
types  of  sensors  -  as  well  as  the  computational  capabilities  underpinning  the  real-time  image  processing  and  sensor 
fusion  systems  -  led  to  prohibitive  costs. 

To  resolve  the  balance  between  cost  and  capability,  the  U.S.  Army  Tank  Automotive  Research,  Development  and 
Engineering  Center  (TARDEC)  -  Ground  Vehicle  Robotics  (GVR)  partnered  with  CERDEC  -  NVESD,  the  U.S.  Army 
Research  Laboratory  -  Human  Research  and  Engineering  Directorate,  and  the  Natick  Soldier  Research,  Development 
and  Engineering  Center  to  establish  a  collaborative  program  to  improve  closed-hatch  vehicle  operations,  mobility 
performance,  and  SA.  This  program,  known  as  the  Improved  Mobility  and  Operational  Performance  through 
Autonomous  Technologies  Army  Technology  Objective  (IMOPAT  ATO),  continues  its  efforts  currently  and  is  intent  on 
developing  a  cost-effective  vehicle-borne  360°  SA  and  indirect  driving  system.  The  IMOPAT  program  will  provide  such 
capabilities  through  electro-optic  indirect  vision,  360°  SA  systems,  threat-cuing  sensors  and  algorithms,  advanced  crew 
stations  functionally  enabled  with  Warfighter-Machine  Interfaces  (WMIs),  and  cognitive  workload  management  and 
monitoring  systems1'8,9.  A  major  projected  outcome  of  the  IMOPAT  ATO  is  to  transition  the  described  capabilities  onto 
the  upcoming  Ground  Combat  Vehicle  (GCV),  Stryker,  and  Mine  Resistant  Ambush  Protected  (MRAP)  platforms. 

In  fulfilling  its  objectives  with  respect  to  visual  SA,  the  IMOPAT  program  will  create  an  affordable  hemispherical 
(360°/90°)  vision  system  with  a  sufficiently  wide  coverage  area,  sensible  ground  intercept  and  up-look  capability,  and 
suitable  range  response.  To  achieve  its  specifications,  the  ATO  will  integrate  a  continuous  360°  field-of-view  system 
using  a  limited  number  of  visual  sensors  onto  a  gigabit  Ethernet  architecture  that  supports  high-definition  (HD)  video 
transmission.  Internally,  the  platform  will  have  three  independently  controlled  workstations.  Despite  enabling  multiple 
workstations,  designers  intend  to  provide  the  capability  to  sustain  360°  SA  and  simultaneous  operation  of  other  vehicle 
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systems  from  a  single  display.  This  functionality  will  be  achieved  with  an  advanced  touch-screen  WMI  that  takes  full 
advantage  of  a  large  in-vehicle  display.  Figure  1  depicts  both  the  external,  vehicle-borne  360°  SA  system  sensors  with 
their  respective  horizontal  field  of  view  and  the  associated  in-vehicle  WMI. 


Figure  1.  Illustration  of  some  of  the  basic  enabling  technologies  underlying  the  360°  situational  awareness  (SA) 
system  being  developed  through  the  U.S.  Army  TARDEC  IMOPAT  ATO  program  (see  text  for  definition  of 
acronyms).  Positioning  and  field  of  view  of  the  external  vehicle-borne  sensors  is  shown  in  A,  while  a  depiction  of 
one  configuration  of  the  in-vehicle  Warfighter-Machine  Interface  (WMI)  is  shown  in  B. 

As  shown,  the  360°  SA  system  will  be  enabled  by  six  fixed  field-of-view  visual  sensors  around  the  vehicle.  Internally, 
the  Warfighters  will  have  a  modularly-organized  display  that  will  allow  for  simultaneous  presentation  of  video  feeds 
from  all  of  the  six  visual  sensors  with  additional  functionality  enabled  by  both  hard  buttons  along  the  lateral  bezel 
positions  as  well  as  soft  buttons  located  underneath  the  video  display.  The  screen  layout  will  be  fully  configurable, 
allowing  the  Warfighter  to  select  to  view  the  sensor  feeds  in  a  variety  of  manners,  from  single  high -resolution  digital 
video  feeds  to  two  vertically  arranged  “banners”  composed  of  up  to  three  contiguous  sensor  feeds  each  (see  Figure  IB). 

As  SA  involves  multimodal  awareness  of  the  environment10,  additional  cuing  technologies  are  to  be  implemented  to 
facilitate  integration  of  auditory  and  visual  sensory  cues.  Specifically,  the  IMOPAT  ATO  will  integrate  sensors  and 
algorithms  to  detect  and  locate  gunfire  in  the  surrounding  environment  and  subsequently  cue  the  Warfighter  via  their 
WMI  display  with  negligible  time-delay.  This  imagery  will  then  be  fed  to  the  in-vehicle  display  allowing  the  operator  to 
interrogate  the  location  with  a  higher-resolution,  gimbal-based  sensor  feed.  Cuing  will  also  be  under  manual  control  and 
enabled  through  the  touch-screen;  when,  for  example,  a  potential  threat  is  noted  in  one  of  the  six  SA  video  feeds,  the 
operator  may  touch  the  associated  video  feed  on  their  WMI  and  the  high-resolution  sensor  will  immediately  slew  to  that 
location.  Finally,  the  ATO  will  demonstrate  the  capability  to  capture  digital  video  from  the  vehicle’s  sensors  onto  an 
integrated  database  and  it  will  demonstrate  the  ability  to  tag  threats  and  log  imagery  within  that  database  for  future 
analysis  as  well  as  immediate  review  of  recent  external  environmental  events.  Warfighters  might  use  this  capability 
either  to  rehearse  missions  or  to  identify  the  locations  of  IEDs  or  other  potentially  threatening  environmental  events. 

3.  HUMAN  FACTORS  CHALLENGES  AND  360°  SA 

Despite  the  remarkable  progress  of  advanced  sensor  systems  such  as  those  being  developed  under  the  IMOPAT  ATO,  as 
well  as  the  current  enthusiasm  for  capabilities  afforded  by  modern  technologies,  significant  questions  remain  regarding 
how  to  best  structure  information  so  as  to  enhance  overall  Warfighter-system  performance.  Of  particular  concern  is  that, 
while  such  technologies  may  provide  expanded  capabilities,  the  broad  array  of  display  options  and  informational 
modalities  that  may  be  provided  threatens  to  overwhelm  human  perceptual-cognitive  abilities,  thus  limiting  the  ability  of 
the  Warfighter  to  effectively  interact  with  targeted  technologies.  Further,  the  additional  tasking  that  follows  on  the  coat¬ 
tails  of  such  technical  advancements  may  lead  to  a  situation  in  which  Warfighters  are  expected  to  manage  an 
increasingly  demanding  and  diverse  array  of  tasks  under  already  stressful  circumstances  and  high-information  loads. 

An  example  of  a  critical  human  factors  question  that  has  received  little  attention  as  engineers  proceed  with  the 
development  of  360°  SA  systems  is  whether,  given  sufficient  quality  and  amount  of  data,  a  human  being  is  even  capable 
of  achieving  a  sustained,  full-spectrum  awareness  of  his  or  her  surrounding  environment,  especially  as  that  environment 
becomes  increasingly  complex  and  dynamic.  That  is,  whether  a  human  is  capable  of  attending  to  and  holding  in  working 
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memory  many  independent  elements  distributed  throughout  a  360°  environment  remains  open  for  investigation. 
Quantitative  data  has  thus  far  shown  that  humans  may  be  limited  in  the  amount  of  information  that  can  be  processed 
cognitively  at  any  given  moment  in  time.  For  instance,  based  on  research  in  fundamental  cognitive  neuroscience,  it  is 
clear  that  humans  are  limited  in  processing  visual  information  in  terms  of  the  distribution  of  targets  throughout 
attentional  space11,  the  number  of  items  that  can  be  held  in  visual  short-term  memory  and  working  memory12,  and  the 
temporal  dynamics  required  to  process  newly-received  visual  information1’.  Together,  these  basic  elements  of  visual 
perception  limit  the  ability  of  an  individual  human  to  actively  maintain  SA  in  a  dynamic  360°  environment. 

Beyond  basic  perceptual  function,  investigators  in  the  domain  of  SA  research  have  identified  cognitive  biases  and 
contextual  factors  that  diminish  the  ability  to  maintain  SA.  For  example,  Endsley,  Bolte  and  Jones14  outlined  8 
“demons”  to  the  achievement  of  SA,  at  least  half  of  which  were  associated  with  what  appear  to  be  cognitive  biases. 
Research  assessing  SA  using  different  display  configurations  in  a  battlefield  context  validated  that,  if  presented  in  certain 
ways,  humans  will  tend  to  exhibit  such  biases.  Specifically,  when  using  a  visual  display  that  presented  environmental 
data  from  an  egocentric  perspective  that  required  manual  panning  to  obtain  full  360°  information,  it  was  shown  that 
Warfighters  “tunneled  into”  particular  aspects  of  the  display  (the  forward  field  of  view)  to  the  detriment  of  attention  to 
areas  located  at  the  periphery15.  This  cognitive  tunneling  was  associated  with  poor  threat  detection  and  was  manifest  in 
reduced  accuracy  and  slow  response  time.  Follow-on  research  indicated  that  switching  the  panning  requirement  from 
one  necessitating  manual  interaction  to  one  that  was  automatic  did  not  facilitate  performance,  but  instead  resulted  in 
emergence  of  the  “out-of-the-loop  syndrome”  wherein  the  operators  became  more  passive  about  and  overconfident  in 
their  acquisition  of  SA  and,  ultimately,  performed  worse  at  threat  detection16. 

Finally,  the  inherent  variability  in  and  associated  instability  of  battlefield  environments  requires  that  the  criterion  for 
successful  system  performance  be  evaluated  against  a  diverse  array  of  operational  conditions.  That  is  to  say,  the 
informational  needs  of  the  operator  are  likely  to  interact  with  the  task  context.  For  example,  what  a  Warfighter  needs  in 
combat  will  certainly  be  different  from  what  is  needed  during  an  extended  surveillance  operation.  Similarly, 
informational  needs  will  vary  depending  on  whether  the  operation  is  to  occur  in  an  urban,  rural,  or  even  a  jungle 
environment.  On  an  even  more  granular  level,  informational  needs  will  vary  within  a  given  operation,  such  as  how  the 
context  differs  between  mission  segments  where  the  vehicle  is  stationary  versus  when  the  vehicle  is  on  the  move.  Or,  at 
the  moment-to-moment  level,  informational  needs  can  fluctuate  in  an  instant  based  on  the  presence  and  recognition  of  a 
particular  object  (i.e.  a  trash  pile  that  appears  to  be  a  location  of  a  potential  IED)  or  due  to  an  environmental  feature  (i.e. 
driving  past  a  previously  difficult -to-see  alleyway),  thus  leading  to  a  temporary  elevation  of  perceived  threat  level. 

Imagine,  for  a  moment,  driving  through  the  fictitious  environment  shown  in  Figure  2,  in  which  the  side-streets  are 
oriented  at  an  angle  to  the  main  axis  of  vehicle  motion  rather  than  having  perpendicular  intersections.  In  such  an 
environment,  something  as  simple  as  the  direction  of  travel  would  likely  impact  our  observations  of  system  use  as  well 
as  perceived  threat  levels.  Specifically,  as  the  vehicle  approached  intersections  in  the  manner  depicted  in  Figure  2A,  we 
note  that  the  most  relevant  vehicle  sensors  for  threat  detection  would  include  the  forward  suite  of  cameras  and  that  the 
operator  would  more  easily  anticipate  upcoming  potential  threats.  For  instance,  the  opportunity  to  observe  and  identify 
whether  a  target  was  a  threat  would  exist  while  approaching,  entering,  and  moving  through  the  intersection. 

However,  in  the  case  of  moving  through  the  exact  same  environment,  but  in  the  opposite  direction  (as  shown  in  Figure 
2B),  we  observe  a  very  different  scenario.  Owing  to  the  angle  of  the  intersections,  it  would  be  impossible  for  the 
operator  to  look  down  the  streets  upon  approach  and,  instead,  he  would  be  constrained  to  examine  potential  threats  only 
after  entering  the  intersection.  More  to  the  point,  in  this  latter  case  any  target  interrogation  would  need  to  occur  using  the 
rear  suite  visual  sensors.  This  also  means  that  the  operator  would  be  forced  to  look  away  from  the  direction  of  primary 
vehicle  motion  in  order  to  maintain  local  area  security.  Considering  the  potential  threat  represented  by  the  gray  shaded 
triangle,  it  would  appear  to  have  greater  opportunity  to  launch  an  attack  on  the  vehicle  in  the  scenario  depicted  in  Figure 
2B  as  opposed  to  that  depicted  in  Figure  2A.  Therefore,  one  would  expect  the  level  of  perceived  threat  to  appear  much 
greater  to  the  operator  (especially  a  trained  and  experienced  Warfighter)  in  Figure  2B.  Again,  this  could  occur  simply  as 
a  function  of  the  direction  of  travel  through  the  environment.  More  to  the  point,  one  would  expect  very  different  use  of 
the  system  in  the  two  environments  shown  in  Figure  2.  In  the  situation  depicted  in  Figure  2A,  one  would  expect  that  the 
operator  would  focus  almost  exclusively  on  the  forward  oriented  visual  sensors  because  that  was  both  the  direction  of 
forward  motion  and  the  direction  in  which  most  new  threats  would  be  likely  to  appear.  In  the  situation  depicted  in  Figure 
2B,  however,  one  might  expect  the  operator  to  either  focus  exclusively  on  the  rear-oriented  visual  sensors  or,  more 
likely,  to  toggle  between  forward-  and  rear-oriented  views. 
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Figure  2.  A  fictitious  environment  in  which  one  might  expect  a  360°  visual  sensor  system’s  use  to  vary  as  a 
function  of  context  and  current  vehicle  motion.  (B)  is  the  same  environment  as  (A),  but  rotated  180°.  Here  the 
shaded  triangle  represents  a  potential  threat  and  all  solid  objects  represent  structures  that  would  occlude  line  of 
sight.  The  vehicle  moving  along  the  main  throughway  is  the  same  as  that  shown  in  Figure  1 . 

In  fact,  recent  evidence  suggests  that  such  context  factors  may  have  a  strong  influence  over  the  use  of  an  actual  360°  SA 
system  in  more  typical  environments.  Specifically,  effects  similar  to  those  discussed  above  were  observed  in  an 
experiment  in  which  operators  were  tasked  with  performing  local  area  security  (visual  scanning  for  threatening  humans 
and  IEDs)  while  riding  in  a  simulated  vehicle  through  a  virtual  urban  environment  and  using  a  prototype  360°  SA  system 
similar  to  the  one  depicted  in  Figure  l17.  Despite  the  fact  that  the  operators  were  provided  with  a  system  enabling 
horizontal  scanning  of  the  full  360°  and  that  they  were  explicitly  instructed  that  they  were  responsible  for  the  entire  field 
of  view,  they  tended  to  focus  their  scanning  efforts  disproportionately  on  the  forward  180°  (and  mostly  the  forward  60°; 
Figure  3).  The  interpretation  of  this  observation,  in  line  with  previous  interpretations  of  similar  results15'16,  was  that 
operators  revealed  a  neurocognitive  bias  called  “cognitive  tunneling”. 
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Figure  3.  Proportional  time  spent  viewing  imagery  from  each  of  the  six  visual  sensors  distributed  around  the 
simulated  vehicle  using  four  different  display  configurations  (see  original  work  for  details17);  darker  shading 
indicates  greater  proportion  of  time  spent  viewing  that  vehicle-relative  direction. 

Irrespective  of  the  difference  between  the  experimental  environment  and  that  used  in  the  example  in  Figure  2  (i.e.  the 
virtual  city  had  perpendicular  intersections),  here  we  interpret  this  result  in  light  of  the  concepts  discussed  above.  It 
seems  that  in  the  presence  of  forward  vehicle  motion,  a  cognitive  bias  might  appear  simply  because  the  vehicle  is 
moving  forward.  This  may  be  especially  true  when  the  task  context  is  one  of  “threat  detection”.  On  a  logical  level, 
approaching  a  potential  hazard  would  seem  more  threatening  than  withdrawing  from  one.  Thus,  in  the  context  of  “threat 
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detection”  while  in  a  vehicle  moving  down  narrow  streets  of  an  urban  environment,  it  would  seem  natural  that  an 
operator  would  focus  attention  in  the  most  task-relevant  direction  -  in  other  words,  the  direction  in  which  new 
information  is  to  be  gained  (that  of  forward  motion)  as  opposed  to  the  direction  in  which  old  information  has  passed. 

Unfortunately,  in  the  case  of  the  study  just  described,  this  cognitive  strategy  was  maladaptive.  This  is  because  threats 
were  scripted  in  such  a  manner  that  they  appeared  for  the  first  and  only  time  in  the  rear  field  of  view  (something  which 
could  easily  happen  in  the  real-world  as  well).  Thus,  when  the  quantitative  data  were  analyzed,  it  turned  out  that  threat 
detection  performance  was  reduced  by  nearly  50%  for  targets  presented  to  the  rear  as  compared  with  targets  presented  in 
front  of  the  vehicle  (Figure  4). 
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Figure  4.  Proportion  of  threats  detected  as  a  function  of  a  set  of  four  different  display  configurations  (see  original 
work  for  details17)  and  as  a  function  of  location  of  threat  onset. 

Overall,  then,  we  have  several  factors  that  must  be  accounted  for  when  considering  the  “human  dimension”  of  the  design 
of  complex  systems.  That  is,  in  addition  to  system  function  and  capability,  we  must  consider  the  limitations,  tendencies, 
and  abilities  of  the  human  operator  as  well  as  the  structure  and  nature  of  the  environment  within  which  the  system  is  to 
be  deployed  and  used.  More  to  the  point,  we  must  consider  the  interaction  between  these  three  aspects  of  the  overall 
“system”  (operator  -  system  -  environment)  and  their  fit  with  one  another.  Therefore,  in  the  context  of  our  current 
discussion  it  can  be  said  that  a  critical  step  in  designing  and  subsequently  evaluating  complex  display  systems  is  to  find 
an  appropriate  match  between  display  capabilities,  operational  needs  as  defined  by  task  and  context,  and  ultimately,  by 
the  Warfighters’  ability  to  utilize  complex  streams  of  dynamically-changing  (and  often  variable  quality)  information. 

4.  AN  UPDATED  FRAMEWORK  FOR  360°  SA  SYSTEM  DESIGN  AND  EVALUATION 

In  the  remaining  portion  of  this  paper,  we  focus  on  the  reciprocity  between  design  and  evaluation  for  360°  SA  systems  to 
be  implemented  in  U.S.  military  vehicles.  In  particular,  we  examine  a  set  of  evaluation  parameters  that  were  originally 
identified  to  link  current  operational  needs  with  technical  specifications  for  the  purpose  of  encouraging  thought  and 
discussion  within  the  military  ground  vehicle  360°  SA  development  community  regarding  how  to  best  design  and  build 
solutions  that  meet  the  needs  of  the  modern  Warfighter.  Rather  than  simply  recounting  what  was  already  presented1, 
however,  we  aim  to  broaden  the  scope  of  this  set  of  evaluation  parameters  to  more  thoroughly  include  and  account  for 
human  factors  considerations  on  the  level  of  behavioral  tendencies  and  fundamental  neurocognitive  function.  Moreover, 
we  couch  our  discussion  in  the  language  of  ecological  psychology  and  cognitive  neuroscience  for  the  purpose  of  relating 
system  properties  to  the  true  nature  of  the  goals  of  design;  that  is,  to  adequately  interface  the  operator,  system,  and  the 
environment  with  one  another  in  the  most  functional  and  adaptive  manner  possible. 

4.1  Characterizing  the  operator-system-environment  interaction  and  situational  awareness 

To  facilitate  our  discussion,  we  first  address  different  ways  of  conceptualizing  the  nature  of  the  relationship  between  the 
operator,  system  and  the  environment  (herein,  we  denote  as  the  O-S-E  fit).  Figure  5  shows  three  of  many  different 
heuristic  models  that  might  be  used  to  represent  the  O-S-E  fit.  Figure  5 A  shows  what  is  meant  to  represent  a  classic 
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model  that  one  might  expect  to  see  from  an  experimental  psychology  (or  even  an  engineereing)  perspective.  In  such  a 
model,  one  would  treat  each  element  as  an  independent  entity,  with  each  having  some  outputs  to  and  accepting  particular 
inputs  from  the  other  elements.  While  sufficient  for  description  of  the  tripartite  interaction  between  the  operator,  system 
and  environment,  such  a  characterization  assumes  a  degree  of  independence  between  the  elements  that  is  not  necessarily 
justified  by  the  real-world  nature  of  the  O-S-E  fit.  Although  this  classic  view  captures  all  pairwise  two-way  interactions, 
it  completely  neglects  the  three-way  interactive  relationship  among  the  elements.  A  second  way  of  conceiving  of  this  set 
of  relationships  is  shown  in  Figure  5B.  This  view,  which  we  may  call  the  ecological  view  (as  it  is  motivated  by  concepts 
from  ecological  psychology),  presents  another  intuitive  and  well -understood  perspective  and,  additionally,  captures  all 
two-  and  three-way  interactions  within  the  O-S-E  fit.  This  view  suggests  that  the  fundamental  structure  of  each  element 
is  composed,  in  part,  of  its  interactions  with  the  other  elements.  Consider,  for  example,  that  systems  are  not  generally 
designed  without  regard  for  the  intended  end-user  or  for  the  environment(s)  within  which  they  would  be  deployed. 


Figure  5.  Three  representations  of  the  relationship  between  the  operator,  system,  and  the  environment,  what  we  call  the  O-S-E  fit.  (A) 
Shows  the  classical  experimental  psychology  (or  engineering)  view,  (B)  represents  the  O-S-E  fit  as  seen  from  the  perspective  of 
ecological  psychology,  and  (C)  shows  a  view  from  what  modern  psychology/neuroscience  considers  embodiment  theory. 


The  third  view  (adopted  herein),  is  motivated  by  modern  concepts  from  both  ecological  psychology  and  cognitive 
neuroscience.  All  systems  are  considered  to  be  “embodied”;  meaning  that  their  function  is  fundamentally  adapted  to  the 
environments  within  which  they  have  been  developed  and,  more  importantly,  their  true  function  only  arises  from  bodily 
interaction  with  that  environment18.  Thus  this  view,  which  we  will  call  the  “embodiment  view”,  has  a  fundamentally 
different  structure  from  the  other  two.  As  depicted  in  Figure  5C,  there  is  a  primacy  of  the  environment  in  the  sense  that 
any  system  is  designed  for  and  ultimately  becomes  a  part  of  the  environment  within  which  it  is  embedded.  Rather  than 
showing  hard  distinctions  between  the  elements  of  the  O-S-E  fit,  we  denote  the  boundaries  between  system,  operator, 
and  environment  with  dash-dot  lines  -  indicating  the  relatively  fluid  nature  of  their  interactions.  Finally,  using  this  type 
of  depiction,  we  can  imagine  a  variety  of  different  configurations  where,  for  example,  the  operator  may  be  completely 
embedded  within  the  system,  sharing  functional  aspects  with  the  system  (‘overlapping  function’)  or,  alternatively,  where 
the  operator  and  the  system  are  completely  separate  but  working  in  parallel  within  a  shared  environment. 


In  Figure  6,  we  represent  the  O-S-E  fit  as  specific  to  the  situation  of  vehicle-borne  360°  SA  systems  (i.e.  the  operator  is 
embedded  within  the  system).  In  addition,  we  map  the  elements  of  the  evaluation  framework  put  forth  by  Mikulski  and 
Berman1,  with  some  minor  revisions,  to  the  elements  of  the  O-S-E  fit.  Upon  first  glance,  we  see  that  the  evaluation 
appears  comprehensive  in  that  there  are  parameters  defined  to  address  each  aspect  of  the  O-S-E  fit.  In  addition,  we  note 
that  this  depiction  is  specific  to  our  situation  in  that  the  operator  only  interacts  with  the  environment  through  the 
intervention  of  the  system  itself;  although,  the  element  of  the  evaluation  that  deals  with  neurocognitive  function 
(neuroergonomic  considerations)  spans  all  three  aspects  of  the  O-S-E  fit,  indicating  that  the  brain  operates  as  an 
intermediary  that  simultaneously  interacts  with  all  sources  of  input  on  behavior  (whether  directly  from  the  environment, 
through  the  system,  or  from  sensory-perceptual  feedback  from  the  operator’s  bodily  action  within  context). 

Importantly,  this  means  of  conceiving  of  the  O-S-E  fit  allows  us  to  specify  more  precise  relations  among  system 
elements  and  the  overall  function  that  they  are  meant  to  subserve  -  that  of  attaining  and  maintaining  dynamic  SA  in  real¬ 
time.  In  general,  S  A  is  thought  to  be  a  function  of  human  neurocognitive  function  and  it  has  been  characterized  as  being 
composed  of  three  inter-related  but  unique  functions  including  (a)  perception  of  the  elements  indicating  the  current 
status  of  the  environment,  (b)  comprehension  of  the  current  status  of  the  environment  and  (c)  prediction  (projection)  of 
the  future  status  of  the  environment10.  The  environment  is  present  as  an  “ambient”  influence  on  the  system  and,  yet  only 
certain  elements  of  its  state  are  detected  by  the  system  sensors. 
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Figure  6.  One  possible  representation  of  the  mapping  between  the  set  of  evaluation  parameters  for  360°  SA 

systems  and  the  totality  of  the  O-S-E  fit. 

In  Figure  7,  we  see  the  direct  flow  of  information  (“data  transmission”  in  the  language  of  Mikulski  and  Berman)  as  solid 
arrows  and  we  see  indirect,  though  intended,  influences  represented  as  dashed  arrows.  As  shown,  the  only  direct 
connection  between  the  operator  and  the  system  is  through  the  in-vehicle  displays  (specifically,  the  Warfighter-Machine 
Interface  that  runs  on  the  in-vehicle  displays)  and  the  only  direct  connection  between  the  system  and  the  environment  is 
through  the  system  sensors.  Intelligent  automations  are  represented  as  being  indirectly  connected  to  the  operator’s 
neurocognitive  system  as  they  are  designed  to  offset  the  above-mentioned  limitations  associated  with  perceptual, 
cognitive,  attentional  and  memory  processes.  Finally,  for  the  sake  of  completeness,  we  included  associations  among  the 
elements  of  neurocognitive  function  that  subserve  SA.  The  forward  (left-to-right)  connections  are  presented  based  on 
logic,  that  is,  one  cannot  comprehend  something  that  they  have  not  perceived  and  likewise  one  is  unlikely  to  make 
adequate  projections  without  at  least  a  minimum  of  comprehension.  The  recursive  connections,  however,  have  also  been 
suggested  by  basic  research  in  psychology  and  cognitive  neuroscience;  for  example,  comprehension  may  affect 
perception  in  a  recursive  manner19  and,  likewise,  projection  (prediction)  has  been  suggested  to  impact  both 
perception"  ’  as  well  as  comprehension""’"  . 

360°  SA  SYSTEM  Neurocognitive  Function  (SA) 


Figure  7.  A  more  detailed  depiction  of  the  O-S-E  fit  representing  the  direct  flow  of  information  (i.e.  data  transmission)  with  solid 
arrows  and  the  indirect  (or  intended)  flow  of  information  with  dashed  arrows. 
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4.2  Evaluation  parameters  for  360°  SA  systems 

Given  the  above  discussion,  we  now  turn  to  a  reconsideration  of  the  evaluation  framework  proposed  by  Mikulski  and 
Berman.  Here,  we  suggest  that  vehicle -borne  360  SA  solutions  will  be  effective  for  modern  combat  applications  as  long 
as  they  adequately  account  for  the  various  elements  of  the  O-S-E  fit.  Factors  to  consider  for  establishing  such 
effectiveness  include  breadth  and  stability  of  system  capability,  abilities  and  limitations  of  the  operator,  and  the  adaptive 
interaction  of  both  the  operator  and  system  with  the  environment.  To  be  adaptive,  we  must  provide  flexibility  to  allow 
for  dynamically  shifting  mission  goals  and  operator  perceptions  (such  as  those  discussed  in  the  example  provided  in 
Figure  2  above).  As  pointed  out  by  Mikulski  and  Berman,  most  of  the  360°  SA  systems  that  have  been  developed  to  date 
share  several  vital  components,  represented  as  “families”  of  evaluation  parameters.  These  include  vehicle-mounted 
sensors  (primarily  visual),  data  transmission  systems,  in-vehicle  displays,  intelligent  automations  (i.e.  cuing  systems), 
and  Warfighter-Machine  Interfaces  (WMIs)  that  should  be  designed  in  accord  with  established  human  factors  principles 
in  order  to  account  for  neuroergonomic  considerations.  We  discuss  each  of  these  families  of  parameters  as  follows. 

Vehicle-Mounted  Sensors.  Sensor  systems  are  the  most  fundamental  component  of  any  vehicle-borne  360°  SA  system. 
Indeed,  as  shown  in  figures  6  and  7,  vehicle-mounted  sensors  often  provide  the  only  interface  between  the  system  (and 
thus  the  operator)  and  the  environment.  Adequately  designed  sensors  enable  effective  detection,  recognition,  and 
identification  of  environmental  entities  from  a  safe  distance.  While  the  primary  focus  of  development  to  date  has  been  on 
visual  sensors,  these  are  often  augmented  by  other  vehicle-mounted  systems  that  sense  the  environment  through  other 
modalities.  Examples  of  additional  modalities  include  acoustic  waves,  sonar,  and  lasers.  Unfortunately,  a  single  visual 
sensor  cannot  address  the  sometimes -conflicting  requirements  of  a  complete  and  broadly  applicable  360°  SA  package. 
For  instance,  military  ground  vehicle  360°  SA  requirements  often  dictate  a  threshold  resolution  for  all  sensors.  Yet,  these 
same  requirements  also  simultaneously  call  for  wide  fields  of  view  and  long  range  characteristics.  At  a  fixed  resolution, 
these  requirements  oppose  one  another;  a  sensor  with  a  wide  field  of  view  inevitably  maintains  a  shorter  range  and  a 
sensor  with  a  long  range  inevitably  maintains  a  narrower  field  of  view.  Of  course,  given  the  variability  in  current  and 
future  theaters  of  operation,  both  characteristics  are  necessary.  Urban  and  jungle  environments  tend  to  benefit  less  from 
range  performance  and  instead  require  broader  field  of  view  while  rural  settings  necessitate  longer  sightlines  and  higher 
resolution  at  distance.  Balancing  these  two  characteristics  in  order  to  meet  all  requirements  through  a  single  sensor  may 
only  be  possible  by  increasing  the  sensor’s  resolution  -  which  may  not  be  technologically  feasible  or  cost-effective. 

As  one  of  the  main  goals  of  Army  modernization  is  to  maintain  agility  and  flexibility  in  its  military  capabilities,  vehicle- 
borne  situational  awareness  systems  must  optimize  for  both  range  performance  and  field  of  view.  An  obvious  solution 
would  be  to  fabricate  an  ideal  sensor  that  could  simultaneously  maintain  long  range  performance  while  preserving  a 
broad  field  of  view.  However,  to  meet  the  conflicting  demands  of  these  parameters,  military  developers  have  instead 
gravitated  towards  layered  system  solutions.  In  the  innermost  layer,  developers  often  place  a  set  of  fixed  sensors  on  the 
vehicle  to  obtain  continuous  360°  horizontal  coverage.  To  reduce  costs  -  and  thus,  the  number  of  components  -  these 
sensors  typically  maintain  a  wide  field  of  view  at  the  necessary  sacrifice  of  range  performance.  As  such,  the  wide  field 
of  view  sensors  are  particularly  suited  for  threat  detection  rather  than  threat  interrogation  and  identification  activities.  To 
achieve  these  latter  capabilities,  engineers  develop  another,  outer  layer  to  the  overall  360°  SA  system  that  includes  high- 
resolution,  narrow  field  of  view  sensors  mounted  on  pan-tilt  (or  gimbaled)  mounts  to  allow  interrogation  of  threats  at  a 
longer  range.  Often,  these  layers  are  augmented  by  an  outermost  layer  that  provides  broad-area  SA  through  networked 
video  communication  with  unmanned  aerial  systems,  unmanned  ground  vehicles,  or  other  military  assets. 

Given  these  issues,  the  primary  evaluation  parameters  that  must  be  considered  for  vehicle-mounted  sensors  include  those 
measurable  parameters  related  to  quantification  of  what  can  be  seen  through  field  of  view  and  range  performance. 
Therefore,  Mikulski  and  Berman  suggested  that  360°  SA  systems  be  evaluated  in  terms  of  simultaneous  field  of  view 
(how  much  of  the  environment  can  be  captured  at  one  time),  sensor  field  of  view  within  a  given  layer  (how  much  of  the 
environment  can  be  captured  by  any  one  sensor  at  a  time),  range  performance  (defined  as  the  maximum  distance  at 
which  a  target  of  a  specific  size  can  be  discriminated),  and  ground  intercept  (providing  for  near-vehicle  SA  capacity). 

Because  the  vehicle-mounted  sensors  dictate  what  information  is  available  to  the  system  and  not  necessarily  the  human 
(i.e.  wide  field  of  view  captured  images  can  be  cropped  or  otherwise  manipulated  for  presentation  to  the  human),  it  can 
be  said  that  the  only  parameter  that  may  relate  to  neurocognitive  function  is  that  of  range  performance.  In  particular,  one 
would  want  to  specify  a  resolution  that,  at  minimum,  supports  later  display  of  images  at  a  resolution  equal  to  or  greater 
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than  that  of  the  receptor  cells  within  the  human  eye  (specifically,  for  the  display  of  images  on  the  region  of  the  retina  that 
has  the  highest  resolution  -  the  central  retina,  where  cells  can  distinguish  stimuli  down  to  minutes  of  the  arc). 

Data  Transmission  Systems.  The  singular  purpose  of  data  transmission  within  360°  SA  systems  is  to  pass  information 
from  one  component  to  another.  As  shown  in  figures  6  and  7,  this  layer  of  the  system  provides  the  interface  from  the 
environment  leading  through  the  system  to  the  operator  via  the  WMI  and  in-vehicle  displays.  Modern  combat  platforms 
typically  use  analog  data  transmission  systems  because  they  tend  to  be  highly  reliable,  easily  integrated  with  the  rest  of 
platform  electronics,  and  perhaps  most  important,  have  a  low  transmission  latency.  Unfortunately,  analog  models 
severely  restrict  the  potential  for  growth  of  modern  technologies  such  as  360°  SA  systems  because  they  tend  to  have 
limited  resolution  and  do  not  allow  video  processing.  As  such,  the  vehicle-mounted  360°  SA  development  community 
has  urged  the  adoption  of  digital  video  architectures  which  provide  greater  resolution  and  allow  for  image  processing, 
which  is  a  key  capability  required  to  augment  visual  displays  through  the  WMI  and  to  thereby  enhance  the  Warfighters’ 
abilities  to  perceive  and  understand  the  environment  surrounding  their  vehicles. 

The  adoption  of  digital  video  architectures  for  360°  SA  systems,  presents  new  challenges  that  must  be  dealt  with.  For 
example,  digital  video  typically  requires  significant  bandwidth  for  data  transmission  and,  as  a  result,  also  exhibits  higher 
transmission  latencies  than  analog  video.  One  computational  solution  to  this  problem,  real-time  video  compression,  has 
not  seen  resounding  success  because  real-time  compression  does  not  typically  reduce  bandwidth  to  sufficient  levels  and 
fiscal  (cost)  overhead.  Yet,  if  360°  SA  system  interfaces  are  to  be  designed  with  computationally  augmented  visual 
displays  (i.e.  for  intelligent  cuing  and  workload  mitigation  discussed  below),  then  there  is  no  choice  but  to  adopt  digital 
architectures.  For  example,  the  pixel-by-pixel  analysis  required  for  intelligent  threat  cuing  and  automatic  target 
recognition  are  difficult  to  achieve  through  analog  video  channels  -  as  there  are  no  pixels  in  analog  video  systems. 

As  should  be  clear  from  the  above  discussion,  then,  the  key  evaluation  parameters  for  the  data  transmission  include 
bandwidth  and  latency.  While  the  previous  section  discussed  vehicle -mounted  sensors  in  a  more  general  way,  here  the 
focus  was  on  transmission  of  video  data  simply  because  it  tends  to  place  the  largest  demand  on  system  bandwidth; 
control  signals  and  other  such  data  tend  to  require  less  bandwidth.  Typical  needs  for  video  transmission  can  range  from 
200  MB/sec  (for  color  VGA  at  a  resolution  of  640  x  480)  to  1.5  GB/sec  (1080p  HDTV  at  a  resolution  of  1920  x  1080). 
Regarding  latency,  one  of  the  biggest  concerns  relates  to  the  neurocognitive  function  of  the  Warfighter  -  that  is, 
qualitative  observations  have  suggested  that  data  transmission  latencies  above  80  ms  (from  sensor  to  in-vehicle  display) 
tend  to  induce  symptoms  of  motion  sickness  and  other  potential  performance  deficits  whereas  below  80ms,  such 
potential  detriments  to  performance  are  not  as  frequently  observed6  8  24 . 

In-Vehicle  Displays.  The  first  level  at  which  we  begin  to  consider  the  operator-system  interaction  involves  what 
information  is  presented  to  the  operator  (Figure  6).  In-vehicle  displays  typically  offer  the  most  natural  interface  between 
a  Warfighter  and  a  vehicle-mounted  360°  SA  system.  Other  informational  modalities,  such  as  tactile  “buzzers”,  exist  and 
their  potential  has  been  demonstrated  in  laboratory  studies25'26.  Yet,  in-vehicle  visual  displays  are  required  to 
comfortably  view  and  interact  with  video  data,  which  is  the  current  standard  for  communicating  information  regarding 
the  external  environment  to  the  vehicle  crew.  Moreover,  in-vehicle  video  displays  are  the  primary  way  in  which  vehicle 
health  indicators  (i.e.  fuel  consumption  and  power  management)  are  communicated  to  the  operators;  thus,  they  are 
particularly  vital  for  any  vehicle -based  SA  system.  In  addition  to  visual  functions,  in-vehicle  displays  are  commonly 
outfitted  with  manual  interface  capabilities  (e.g.  touch  screens)  that  are  augmented  by  hard  bezel  buttons  along  their 
edges  as  shown  in  Figure  IB.  Caution  and  significant  human  factors  engineering  efforts  are  required  when  attempting  to 
develop  such  interfaces  through  the  touch  screen  capability  alone,  however,  because  the  graphic  features  involved  will 
invariably  require  additional  physical  space  within  the  WMI  and  they  may  unintentionally  obscure  important  video  data. 

Because  the  in-vehicle  display  of  data  is  such  a  vital  component  of  the  vehicle-mounted  360°  SA  system,  its  parameters 
cannot  be  at  all  disconnected  from  the  structure  of  the  360°  SA  system  itself.  Using  these  considerations  and  assuming 
video  as  the  primary  informational  modality,  Mikulski  and  Berman  thus  identified  screen  size,  screen  resolution  and  the 
visibility  settings  (e.g.  brightness  and  contrast)  as  the  set  of  evaluation  parameters  for  such  system  components. 
However,  from  a  neurocognitive  standpoint,  it  is  worth  noting  that  current  thinking  indicates  that  human  information 
processing  capacity  within  one  sensory  channel  (i.e.  vision)  may  be  easily  overloaded.  Therefore,  further  investigation  of 
how  to  use  alternative  modalities,  such  as  the  tactile  “buzzers”  mentioned  above,  is  not  only  warranted  but  in  fact  may 
become  critical  as  the  density  of  information  to  be  managed  in  complex  military  scenarios  increases  with  each  new 
technology  added.  Indeed  research  has  shown  that  it  is  possible  to  surpass  the  so-called  “attentional  bottlenecks”  created 
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when  attempting  to  overuse  a  single  modality  by  “substituting”  sensory  information  from  alternative  sources27.  As  such, 
data  “display”  (presentation)  evaluation  parameters  will  need  to  co-evolve  with  the  integration  of  additional  modalities 
such  as  auditory  and  tactile  informational  streams. 

Intelligent  Automations.  Sensors  that  operate  on  the  vehicle -mounted  360°  SA  systems  collect  vast  amounts  of  data  and 
often,  humans  are  restricted  in  their  ability  to  effectively  analyze  that  information  and  simultaneously  perform  other 
mission-critical  operations.  Thus,  significant  Army  research  and  development  efforts  have  focused  on  analysis  of  and 
techniques  to  reduce  or  mitigate  cognitive  load  on  Warfighters  as  they  operate  new  technologies  such  as  a  vehicle- 
mounted  360°  SA  system  ’  ’  .  In  addition  to  workload  reduction,  the  information  collected  from  vehicle-mounted 
sensors  may  also  be  analyzed  to  cue  Warfighters  to  potential  threats  to  their  safety.  For  example,  such  technologies 
could  draw  attention  to  potential  enemy  combatants  following  real-time  analyses  by  an  automated  target  recognition 
system  (or  another  human  operator)  or,  in  a  similar  manner,  could  inform  Warfighters  of  potential  IEDs  identified  by 
analyses  of  previously  recorded  digital  video.  Similar  applications  could  be  applied  for  vehicle  control,  where 
automations  could  be  used  to  identify  road  edges  or  traversable  off-road  terrain  to  mitigate  vehicle  rollover  risks  as  in 
safeguarded  teleoperation30.  As  was  introduced  in  the  discussion  of  in-vehicle  displays,  the  multisensory  nature  of 
human  perception  could  also  be  exploited.  For  example,  cuing  technologies  could  be  multi-modal,  perhaps  in  notifying 
Warfighters  of  threats  with  auditory  and/or  tactile  alerts. 

Unfortunately,  intelligent  cuing  technologies  are  often  rendered  unreliable  because  they  analyze  information  from  noisy 
sensors  in  dynamic,  unstructured  environments  for  which  their  underlying  mathematical  or  statistical  models  might  not 
have  been  formulated.  Consequently,  intelligent  cuing  often  maintains  a  high  rate  of  false  alarms  combined  with  low 
probabilities  of  correct  detection1.  Therefore,  it  becomes  highly  likely  that,  faced  with  the  choice  of  basing  decisions  on 
demonstrated  unreliable  information.  Warfighters  would  ignore  or  disable  such  automations  for  the  more  proven 
techniques  of  relying  on  their  own  eyes  and  ears.  Indeed  such  disuse  behaviors  have  been  shown  in  other  contexts  of 
human  use  of  automation3".  Therefore,  any  effort  to  transition  intelligent  cuing  technologies  to  fielded  vehicle  systems 
must  account  for  critical  usability  issues. 

Additionally,  intelligent  cuing  technologies  often  incur  heavy  computational  loads.  The  need  to  analyze  large  amounts  of 
data  and  to  run  computationally-intensive  algorithms  in  real-time  dictates  a  concurrent  need  for  an  integrated  digital  data 
transmission  system.  Recalling  our  prior  discussion  of  data  transmission  requirements  and  the  heavy  reliance  on  digital 
video  architectures  in  360°  SA  systems,  the  computational  cost  of  such  algorithms  must  also  be  weighed  in  light  of  the 
overall  latency  of  the  360°  SA  system.  That  is,  such  algorithms  may  drive  total  latencies  beyond  acceptable  limits, 
unbalancing  an  already  tenuous  relationship  between  bandwidth  utilization  and  information  presentation. 

Despite  current  challenges  to  their  implementation,  the  military  still  believes  that  investing  in  intelligent  cuing 
technologies  can  provide  enormous  potential  benefits  to  combat  and  other  operations.  Thus  we  consider  their  inclusion 
in  360°  SA  systems  of  the  future  and  therefore,  they  must  be  evaluated  against  reasonable  parameters.  As  shown  in 
Figure  6,  we  note  that  although  intelligent  cuing  is  a  technological  capability  provided  by  the  system,  assessment  is 
largely  driven  by  the  needs  of  the  operator  with  respect  to  its  usability.  Thus,  considering  the  cognitive  behavior 
potential  of  Warfighters  with  respect  to  intelligent  automations  (i.e.  the  potential  for  disuse  or  even  misuse  of  the 
technology)  the  identified  evaluation  parameters  include  estimators  of  overall  reliability  and  validity  of  system  outputs 
(primarily  we  look  to  probability  of  correct  detection  and  false  alarm  rates)  as  well  as  systems -level  issues  such  as 
computational  efficiency  and  feasibility  of  model  implementation  in  software. 

Neuroergonomic  Considerations.  As  described  throughout  this  discussion,  vehicle-mounted  360°  SA  systems  are 
incredibly  complex  and  the  cognitive  loads  required  of  Warfighters  during  the  analysis  and  control  of  360°  SA 
subsystems  must  be  reduced  or  managed  through  the  development  of  effective  Warfighter-Machine  Interfaces  (WMIs). 
WMIs  are  often  designed  for  in-vehicle  displays  to  control  and  analyze  information  from  vehicle-mounted  visual 
sensors,  but  other  modalities  may  be  employed  to  provide  primary  or  redundant  capabilities  alongside  in-vehicle 
displays.  For  instance,  yolks  or  keyboards  may  be  utilized  to  control  pan-tilt  mechanisms  or  audible  messages  may  be 
developed  to  provide  redundant  threat  cuing  and  localization  capabilities.  WMIs  may  be  built  in  various  manners  but 
they  must  be  developed  in  accordance  with  established  human  factors  principles  that  simplify,  or  at  least  lend  efficiency 
to,  a  Warfighter’s  interaction  with  the  vehicle-mounted  360°  SA  system. 

One  of  the  challenges  faced  in  the  design  process  for  evolving  military  systems  is  uncertainty  about  how  well  they  will 
provide  for  the  complex  cognitive  operations  that  the  Warfighter  must  execute  during  risky  and  time -pressured  missions. 
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In  general,  advanced  systems  are  designed  by  software,  systems,  and  human  factors  engineers  based  on  a  priori 
assumptions  or  on  a  posteriori  assessments  of  usability  in  specific  environments.  Frequently,  such  assessments  are  based 
on  tasks  that  are  either  too  limited  to  be  operationally  relevant  or  are  too  complex  to  allow  for  precise  scientific  inference 
regarding  generalized  behavioral  tendencies.  Moreover,  such  assumptions  and  assessments  often  neglect  consideration 
of  how  humans  naturally  process  information  within  their  neurocognitive  system.  The  typical  assumption  underlying 
system  development  is  that,  other  than  providing  assurances  for  functionality,  the  uncertainties  regarding  usability  can  be 
countered  by  allowing  sufficient  time  for  operators  to  learn  how  to  use  the  system.  Yet,  one  objective  of  military 
development  is  to  create  interfaces  for  which  the  time  required  to  train  operators  is  minimized33. 

Neuroergonomics  is  an  emerging  area  of  investigation,  spawned  by  research  and  theory  in  disciplines  such  as 
ergonomics  and  neuroscience,  in  which  the  objective  of  research  is  to  understand  how  the  human  brain  functions  in 
complex,  real-world  environments34.  Neuroergonomic  design,  then,  is  centered  on  creation  of  tools  and  interfaces  that 
naturally  compliment  the  user,  using  design  principles  rooted  in  the  natural  processing  capabilities  of  the  neurocognitive 
system.  Recently,  the  U.S.  Army  has  taken  interest  in  the  integration  of  theory  and  practices  from  cognitive  neuroscience 
and  neuroergonomics  in  that  they  may  lead  to  development  of  systems  that  will  improve  and  optimize  Warfighter- 
system  performance  in  real  operational  scenarios  while  minimizing  training  time  needed  to  gain  proficiency33. 

Fortunately,  considerable  research  in  human  factors,  psychology  and  cognitive  neuroscience  have  brought  about  the 
development  of  standard  metrics  and  advanced  the  application  of  mathematical  modeling  to  assess  the  effectiveness  of 
WMIs  in  terms  of  human  behavior  and  neurocognitive  function.  Many  of  these  metrics  and  models  aim  to  determine  the 
ease  and  quickness  with  which  a  Warfighter  interacts  with  system  capabilities,  as  well  as  assessing  and  understanding 
how  different  display  and  informational  conditions  impact  the  function  of  the  operator’s  nervous  system.  Traditionally, 
such  assessments  required  verification  through  extensive  qualitative  user  evaluations  that  were  subject  to  variations  in 
individual  tastes,  interests,  and  capabilities.  However,  more  recent  Army  efforts  have  begun  to  bring  scientific  rigor  back 
into  the  assessment  of  complex  systems  in  complex  environments35.  As  the  science  progresses  in  this  direction,  it 
becomes  increasingly  possible  to  add  to  our  understanding  of  how  to  evaluate  the  effectiveness  of  WMI  for  complex 
military  applications.  Examples  of  evaluation  parameters  that  would  prove  useful  include,  but  are  unlikely  to  be  limited 
to,  measures  such  as  probability  of  correct  threat  identification,  optimality  of  movement  parameters  required  for 
interface  interaction  (i.e.  movement  time  vs.  reaction  time),  time  to  comprehend  presented  information,  optimality  of 
speed  and  accuracy  of  decision  making,  as  well  as  assessments  that  require  more  advanced  data  collection  and  analysis, 
such  as  understanding  how  to  induce  recruitment  of  proper  task-specific  brain  regions  and  maintain  adequate 
physiological  levels  (i.e.  arousal)  and  attentional  levels  to  optimize  mission  performance. 

CONCLUSION 

In  this  paper,  we  have  presented  and  discussed  the  complex  topic  of  providing  display  systems  adequate  to  support  the 
acquisition  and  maintenance  of  360°  situational  awareness  (SA)  in  dynamic  military  vehicle  environments.  In  so  doing, 
we  have  reviewed  two  examples  of  systems  that  have  been  developed  by  U.S.  Army  scientists  and  engineers,  alongside 
industry  partners,  one  of  which  is  still  the  focus  of  a  current  Army  program  (TARDECs  IMOPAT  ATO).  In  order  to 
motivate  our  discussion  of  evaluation  parameters  suitable  to  stand  as  criterion  for  assessing  the  quality  and  capability  of 
360°  SA  systems  as  they  are  matured,  we  then  discussed  a  variety  of  human  factors  challenges  to  the  systems 
developers.  The  ultimate  point  of  focus  for  this  discussion  was  to  emphasize  the  need  to  take  a  broad  view  on 
understanding  how  to  design  an  optimal  system  to  meet  the  needs  of  the  modern  Warfighter;  that  is,  we  concluded  that 
the  need  rests  in  optimizing  the  fit  between  display  capabilities,  operational  needs  as  defined  by  task  and  context,  and 
ultimately,  by  the  Warfighters  ability  to  utilize  complex  streams  of  dynamically-changing  information  -  in  the  shorthand 
of  the  current  paper,  we  called  this  the  operator-system-environment  fit.  In  the  end,  we  reviewed  and  revised  a  set  of 
evaluation  parameters  that  were  previously  put  forth  to  unite  all  interested  stakeholders  under  a  common  framework  to 
ensure  that  360°  SA  systems  continue  to  provide  Warfighters  with  the  ability  to  generate  sound  decisions  in  combat. 
Informed  by  elementary  concepts  and  data  from  studies  in  human  factors,  psychology  and  cognitive  neuroscience,  our 
overall  aim  was  to  enhance  the  understanding  of  the  relatively  critical  nature  of  considering  the  interaction  between  the 
operator,  system  and  environment  as  the  essential  ingredient  to  advancing  systems  of  sufficient  quality  to  enhance 
lethality  and  survivability  of  the  Warfighters  of  today  and  those  of  years  to  come. 
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